The Windows test failures are unrelated. I'll try to fix them later in the day, but happy to step back if someone else does it before me, or wishes to.

To correct my above comment - I was confused about the version of pyodide-build versus pyodide itself - it seems that the option CIBW_PYODIDE_VERSION is still necessary, but that the ability to configure the package pyodide-build might also be useful, though is better done through CIBW_DEPENDENCY_VERSIONS.

It looks like the next step here is to make CIBW_PYODIDE_VERSION a proper option, with documentation. Would you like assistance with that @agriyakhetarpal ? I can certainly help with the options spec/parsing bit, though I think you might be better placed to do the documentation bit, there might be nuance/guidance I'm unaware of.

I've pushed a change to make the option read properly through options.py, and it removes some hardcoding of pyodide-build and emscripten - pyodide-build is already spec'd in dependency-version constraints, and an emscripten version can be read from pyodide-build's output at runtime.

Still to do -

• options TOML schema update
• tests
• docs for new option pyodide-version.

Added docs and tests. Ideas welcome for a way to test pyodide-version itself! Ideally we'd assert that the package is built with the specific version. That said I'm happy enough with the current coverage.

@agriyakhetarpal, I hope you don't mind me taking a run at this! I was reviewing it again and found myself forming opinions about how the versions are pinned, so I wanted to see if it worked.

Another thing that I noticed is that pyodide version updates aren't automated yet. pyodide xbuildenv search --json should be a fairly easy way to automate that in bin/update_pythons.py.

Hi @joerick, thank you so much! Also, apologies for the radio silence here – I couldn't take a look in January, but I'm happy to see it through! #2122 looks like it was a beneficial improvement.

I have a few comments that we should take a look at. I haven't gone through the new code changes you pushed fully, so I apologise if these have already been resolved in some form. Here are the primary blockers on this PR that I had previously noticed when I was working on it more actively late last year:

• pyodide-build has a strict requirement on the Python version being used. For example, the cibuildwheel action sets up Python 3.12 to be compatible with Pyodide 0.27, so it's fine to use CIBW_PYODIDE_VERSION with a newer Pyodide version which also has the same CPython version (we usually try to update to a new CPython version after eighteen months – it's usually @hoodmane who takes it up). However, this wouldn't work with Pyodide 0.28, which will ship with CPython 3.13 (see Python 3.13 pyodide/pyodide#5498). Thus, setting CIBW_PYODIDE_VERSION to a newer one will break cibuildwheel. There are two ways to resolve this:
  • either make cibuildwheel somehow aware of what version of setup-python it should set up for Pyodide, based on a precomputed list of Pyodide vs CPython versions that can be either maintained on the cibuildwheel side or on the Pyodide side (perhaps we can include a Python version key in https://github.com/pyodide/pyodide/blob/main/pyodide-cross-build-environments.json that cibuildwheel can just read?);
  • or, make it possible on the Pyodide tooling side to cross-compile from one CPython version to another Pyodide's CPython version, i.e., relax pyodide-build's Python version requirement and make it support compiling to cp312-pyodide_wasm32 from, say, when it's installed in CPython 3.13. This is the same limitation as in https://github.com/benfogle/crossenv and is more challenging to resolve.
• I wonder if we should allow building for multiple Pyodide versions at a time, similar to PyPy. One of the goals I've been working on over the past year is using the current Pyodide support in cibuildwheel to build nightly wheels for use in interactive documentation deployments. So, if there's a JupyterLite deployment that uses a specific Pyodide version, say 0.27, and the package maintainers update cibuildwheel to one that supports Pyodide 0.28, nightly/dev docs deployments would break because Pyodide 0.28 would build wheels for a new Pyodide ABI (which wouldn't be compatible with Pyodide 0.27 deployed in the docs job). If cibuildwheel were to build wheels for both Pyodide 0.27 and Pyodide 0.28 (and keep adding new Pyodide versions as they release) and allow skipping a particular Pyodide version using its identifier through CIBW_BUILD and CIBW_SKIP options, that would make interactive docs more reliable. We could document this behaviour more notably, noting that building multiple Pyodide wheels is a Pyodide-specific case and users should explicitly set what version(s) to build/skip. I've also discussed this aspect here: CI Move Pyodide CI from Azure to GitHub Actions scikit-learn/scikit-learn#29791 (comment)

pyodide-build has a strict requirement on the Python version being used

I think @ryanking13's plan is that we will relax this, in the sense that we will continue supporting Python 3.12 in pyodide-build even after we upgrade to using Python 3.13. But what does always need to be guaranteed is that target Python version == build Python version. So if we're building a wheel with abi tag pyodide_2025_0 for Pyodide 0.28, the build machine needs to use Python 3.13. If we're building a wheel with abi tag pyodide_2024_0 for Pyodide 0.27 and 0.26, then the build machine needs to use Python 3.12.

pyodide-build has a strict requirement on the Python version being used

Ah, interesting. I didn't know that! This isn't too tricky, I think. The way I'd suggest to approach this is to read the python version we need from pyodide xbuildenv search --json (the python version is already listed there, as well as pyodide-cross-build-environments.json) and install it from astral-sh/python-build-standalone. Q: I assume we don't need to worry about patch versions here?

Previously we've avoided using 3rd-party distributions of CPython, for fear of producing binaries with poor compatibility, but in this case we only need it to run the build, there's no implicit linking going on, right?

I wonder if we should allow building for multiple Pyodide versions at a time, similar to PyPy

In cibuildwheel lingo, this would amount to putting the Pyodide version into the build identifier. Aside: we don't actually do this for PyPy, we're only building the latest PyPy version per Python minor version, even if there are multiple PyPy ABIs within each minor. That doesn't mean we couldn't do it for Pyodide.

if there's a JupyterLite deployment that uses a specific Pyodide version, say 0.27, and the package maintainers update cibuildwheel to one that supports Pyodide 0.28, nightly/dev docs deployments would break because Pyodide 0.28 would build wheels for a new Pyodide ABI (which wouldn't be compatible with Pyodide 0.27 deployed in the docs job).

I've been skimming @hoodmane's draft PEP 776 re. emscripten. Wouldn't a pyodide_2025_0 wheel be forward compatible with a version of Pyodide that is released later? I found this in the draft PEP:

In order to balance the ABI stability needs of package maintainers with the ABI flexibility to allow the platform to move forward, Pyodide plans to adopt a new ABI for each feature release of Python.

If that's the case, (i.e. a 1:1 mapping between Python minor version and wheel ABI) I think keeping the build identifier tied to the Python minor version should suffice. Please correct me if I'm missing something though!

Q: I assume we don't need to worry about patch versions here?

That's correct, we do not need to worry about patch python versions.

I think @ryanking13's plan is that we will relax this, in the sense that we will continue supporting Python 3.12 in pyodide-build even after we upgrade to using Python 3.13. But what does always need to be guaranteed is that target Python version == build Python version.

Yes, exactly.

I think keeping the build identifier tied to the Python minor version should suffice. Please correct me if I'm missing something though!

If a package uses numpy or scipy at build time, it may be sensitive to the specific Pyodide version and not just the Python minor version. But only insofar as it depends on a specific numpy/scipy version, and this dependency should be clear from its Requires-Dist information. So I agree that putting the Python minor version in the build identifier will suffice.

Previously we've avoided using 3rd-party distributions of CPython, for fear of producing binaries with poor compatibility, but in this case we only need it to run the build, there's no implicit linking going on, right?

That's right, if pyodide-build is functioning correctly we shouldn't be using any headers or libs from the build Python.

Wouldn't a pyodide_2025_0 wheel be forward compatible with a version of Pyodide that is released later?

Yes, assuming that we first determine the pyodide_2025_0 ABI and implement it in pyodide-build and then release pyodide-build. The pyodide_2025_0 isn't stable yet though so currently it's not a good idea to distribute wheels with that platform tag except for experiments.

Thanks for the responses @hoodmane and Pyodide folks!

So I think the next thing to do would be to remove the implicit reliance on the host Python version, perhaps with python-build-standalone. That can be a follow-up PR, no need to add that here.

The pyodide_2025_0 isn't stable yet though so currently it's not a good idea to distribute wheels with that platform tag except for experiments.

That's cool, I was speaking hypothetically, as in, "once the ABI is stable".

If a package uses numpy or scipy at build time, it may be sensitive to the specific Pyodide version and not just the Python minor version.

Just so I understand this- is that because pyodide bundles these libraries? And is this just a build-time concern or would that also limit the compatibility of the built wheels?

is that because pyodide bundles these libraries?

Yes.

And is this just a build-time concern or would that also limit the compatibility of the built wheels?

I don't think it should limit compatibility of the built wheels beyond what they already say in their Requires Dist. If the wheel says it wants scipy >= 1.7 for instance then I think that is an assertion by the wheel that it works the same with scipy 1.7 and scipy 1.8 and can be build with either unless it has a more specific build_requires. If the wheel built against scipy 1.7 isn't compatible with scipy 1.8, then I think it's on the wheel to pin scipy==1.7, which would make it only compatible with Pyodide versions that bundle scipy 1.7. I don't think Pyodide specifically introduces any new limitations or special considerations here.

Based on these recent discussions, here's what I understand and propose:

• We will continue to have the requirement/limitation of the xbuildenv/host Python version being the same as the Pyodide Python version for pyodide-build to operate.

  • so, would the idea be that we'll download a Python binary from python-build-standalone in cibuildwheel/platforms/pyodide.py, install it, install pyodide-build in a virtualenv with it as the creator (similar to how macOS downloads CPython binaries), and compile the requested package to WASM – and we can get what Python version we need to install for whatever is supplied to CIBW_PYODIDE_VERSION: using pyodide xbuildenv search --json --all?
  • the idea is that the PR wouldn't be usable without that, as the cibuildwheel GitHub Action won't be able to build against 0.28 when it lands, or even any nightly xbuildenv of Pyodide shall we implement grabbing it, as we've updated much later from Emscripten v3.1.58 and now bumped to Python 3.13 a few moments ago as well. This is a bit unfortunate, considering how convenient the GitHub Action is. However, it should be usable if someone were to do python3.13 -m pip install cibuildwheel && cibuildwheel --platform pyodide, so maybe we should document this case – i.e., don't use the action or any other appropriate note?
  • or, should we add the pyodide xbuildenv search --json --all logic to the GitHub Action instead (perhaps through a pipx step) so that it picks up the Python version needed for the requested Pyodide version (if a build for Pyodide is requested, that is, otherwise not) and then passes that along as an input to setup-python? It makes the action a bit more complex, but none of it is exposed that much to the user anyway and is probably minimal enough to incorporate.

• Please feel free to push back on this thought, however, IMO, it's more elegant to do this:

   steps:
     - uses: pypa/[email protected]
       env:
         CIBW_PLATFORM: pyodide
         CIBW_BUILD: "pyodide_2024_0 pyodide 2025_0"
         CIBW_TEST_REQUIRES_PYODIDE: "<...>" # and so on
         ...

  rather than to do this:

   steps:
     - uses: pypa/[email protected]
       env:
         CIBW_PLATFORM: pyodide
         CIBW_TEST_REQUIRES_PYODIDE: "<...>" # and so on
   
     - uses: pypa/[email protected]
       env:
         CIBW_PLATFORM: pyodide
         CIBW_PYODIDE_VERSION: "0.XY"
         CIBW_TEST_REQUIRES_PYODIDE: "<...>" # and so on
         ...